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Soft Control on Collective Behavior of a 
Group of Autonomous Agents by a Shill Agent 

Jing Han, Ming Li and Lei Guo 



Abstract 

This paper asks a new question: how can we control the collective behavior of self-organized multi- 
agent systems? We try to answer the question by proposing a new notion called 'Soft ContwV , which 
keeps the local rule of the existing agents in the system. We show the feasibility of soft control by 
a case study. Consider the simple but typical distributed multi-agent model proposed by Vicsek et al. 
Q ' for flocking of birds: each agent moves with the same speed but with different headings which are 

updated using a local rule based on the average of its own heading and the headings of its neighbors. 
►^ . Most studies of this model are about the self-organized collective behavior, such as synchronization of 

^Ij ■ headings. We want to intervene in the collective behavior (headings) of the group by soft control. A 

OO ' specified method is to add a special agent, called a 'ShiU\ which can be controlled by us but is treated 



as an ordinary agent by other agents. We construct a control law for the shill so that it can synchronize 
the whole group to an objective heading. This control law is proved to be effective analytically and 
numerically. Note that soft control is different from the approach of distributed control. It is a natural 
way to intervene in the distributed systems. It may bring out many interesting issues and challenges on 
the control of complex systems. 
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I. Introduction 

Collective behavior is the high level (macroscopic) property of a self-organized system which 
consists of a large number of (microscopic) individuals (agents). Examples are synchronization, 
aggregation, phase transition, pattern formation, swarm intelligence, fashion, etc. People found 
this kind of phenomena in many systems, such as flocking of birds, schools of fish, cooperation 
in ant colonies, panic of crowds [10], norms in economic systems [9], etc. Without any question, 
collective behavior is one of the fundamental and difficult topics of the study of complex systems. 
We classify the research on collective behavior into three categories. 

(I) Given the local rules of agents, what is the collective behavior of the overall system? 
Many people have been working on this category which is about the mechanism of how collective 
behavior emerges from multi-agent systems. "More is different"[l]. The physicists have applied 
theory of statistical physics to explored some simple models, from the Ideal Gas model. Spin 
Glasses, to the panic model and network dynamics. 

(II) Given the desired collective behavior, what are the local rules for agents? Some 
people work on this category. One typical example is Swarm Intelligence. Since the high level 
function of the overall system can be more than the sum of all individuals, how do we construct 
robust intelligence by a large number of locally interacting simple agents? An ant is simple and 
often moves randomly. But a colony of ants can efficiently find the shortest path from their nest 
to a food source. This natural phenomenon inspired the Ant Colony Optimization Algorithm[2]. 

(III) Given the local rules of agents, how we control the collective behavior? In some real 
applications, it is very difficult or even impossible to change the local rules of agents, such as 
the behavioral rules of people in panic and the flying strategies of birds. Yet we need to control 
the system to avoid danger or improve efficiency. Then what is the feasible way to intervene in 
the collective behavior? There should be a way especially for the multi-agent system that utilizes 
the property of collective behavior. This question is not well noticed in the control literature. 
In this paper, we propose a new control notion, called 'soft controlu, which keeps the basic 



local rule of the existing agents in the system. There are no global parameters can be adjusted 

'The idea of soft control in this paper was inspired by discussions at the "Systems, Control and the Complexity Science" 
Xiangshan Science Conference (May, 2004, Beijing) and the preliminary result was presented at the 2nd Chinese-Swedish 
Conference on Control [13]. 
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to achieve the control purpose, such as adjusting the temperature to change water from liquid 
phase to gas/solid phase or broadcasting orders directly to all agents by a controller. This feature 
makes a difference between soft control and the traditional control. 

These three categories are tightly related. The first category (I) is about how a distributed 
system behaves and what emerges from the system. Both of (II) and (III), however, require the 
system to behave at the high level as what we expected, i.e., we know what collective behavior 
should be emerged from the system. But (II) focuses on how to design a distributed system by 
constructing(changing) the local rules for interaction among agents, and (III) focuses on how to 
intervene in the existing system by a nondestructive way. 

Since collective behavior is a kind of macroscopic feature of a multi-agent system, usually 
adding one or a few more agents will not affect it. But if those agents are special ones which can 
be controlled, even though only a small number (in some cases, only one), it can dramatically 
change the collective behavior as will be shown in Section 3. So in this paper, a way to control 
the system without changing the local rules of the existing agents is to add one (or a few) special 
agent which can be controlled. Therefore its behavior does not necessarily obey the local rules 
as the ordinary agents do. However, the existing agents still treat the special agent as an ordinary 
agent, so the special agent only affects the local area with limited power. The special agent is the 
only controlled part of the system and it indeed changes the collective behavior of the system 
by 'cheating' and 'seducing' its neighboring ordinary agents. So we do not call the special agent 
a leader. Instead, we call it a 'ShilV . 

Soft control is different from distributed control. Distributed control recently gets more and 
more attention because many real-world applications are distributed systems, such as power 
networks and traffic systems. The system consists of many interacting subsystems that all have 
their own controllers with local feedback. The study of the relationship about performance 
between the overall system and the subsystems falls into category (I). On the other hand, 
designing a practical distributed system for resource sharing and cooperation is a demanding and 
complicated task. This research is of the category (II). So we can see that both soft control and 
distributed control concern with the macroscopic collective behavior (such as synchronization) 
of the self-organized multi-agent system with local rules. But in distributed control, every agent 
is treated as a control system and has its control law (which is the set of local rules); while 
in soft control framework, all these agents are treated as one system and the control law is for 
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the skill. So soft control can be regarded as a way of intervention in the distributed systems. 
With the growing literature on complex systems, the challenge of controlling complex systems 
is likely to become a key problem for control scientists. 

This paper is going to explore the idea of soft control by a case study, where we will show 
how a skill can be used to control the headings of a group of mobile agents demonstrated based 
on the modified Bold model [4] proposed by Vicsek et al [3]. The current control theory cannot 
be applied directly to this problem because our soft control is a nondestructive intervention in 
a locally interacting autonomous multi-agent system, which is not allowed to change the local 
rules of the existing agents. The formation control [5] mainly focuses on how to design the 
local rules of a team of robots to maintain a geometric configuration during movement, which 
is a very special kind of collective behavior, and its approach belongs to distributed control. 
The pinning control [12] is especially for stabilizing dynamical networks (with special topology, 
such as random networks, small-world and scale-free networks) by imposing controllers on a 
small fraction of selected nodes in the network. No one appear to have directly studied the above 
mentioned soft control problem in the literature. Recently, Jadbabie et al [6] investigated a multi- 
agent model which is slightly different from the one proposed by Vicsek et al, they showed that 
under some a priori connectivity conditions, the group will eventually move in a same direction 
even without centralized control. This is a problem of the category (I) of collective behavior 
study. An idea in [6] worthy of mention is the Leader Following model, which has a leader 
agent with fixed heading. But it did not tell how to implement synchronization by a leader. The 
so-called 'virtual leader' [7] is not treated as an agent but part of the potential, which requires 
to introduce new rules for the ordinary agents to recognize it. 

The rest of this paper is organized as follows: The modified BOID model proposed by Vicsek 
et al and the notion of soft control are set up in Section 2. Section 3 is a case study for soft 
control. We will solve a simple but non-trivial problem by constructing an effective control law 
for the skill. Both theoretical analysis and computer simulation show that one skill is possible 
to turn the direction of the whole group. Some concluding remarks are given in Section 4. 

IL Soft Control of the Modified Boid Model 

Eighteen years ago, Renolds proposed a simultaneous discrete-time multi-agent model, which 
is called Boid[4], to catch the natural phenomenon of flocking of birds and fish and make 
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computer animation. In this model, each bird decides its flying direction only by looking 
at its current neighboring birds and using three rules (Alignment, Separation and Cohesion) 
based on the status of its neighbors. These rules are local and simple but the overall system 
exhibits the flocking behavior. The Bold model became very popular in complex systems studies. 
Unfortunately, this simple model is not simple at all for theoretical analysis. 

In 1995, to investigate the emergence of self-ordered motion, Vicsek et al [3] proposed a 
model which is actually a modified version of Bold, by only keeping the Alignment rule- steer 
towards the average heading of neighbors, which still can catch the flocking behavior. There are 
n agents (xj(-),^j(-)) for n birds, labelled from 1 through n, all moving simultaneously in the 
two-dimensional space. The velocity of agent i at time t is defined by 

Viit) = {v cos{9i{t)),v sm{9i{t))) (1) 

which is constructed to have an absolute value v and a heading given by the angle 9i{t) E [0, 27r). 
The position of agent i at time t is denoted as JCj(t). The neighborhood of agent i at time t 
is defined as Ni{t) = {j|||jCj(t) — jCi(t)|| < r,j = 1,2, ... ,n}, where r is the radius of the 
neighborhood circle. For any agent i, its heading and position are updated by ^ and ([3]) below 
if ignoring noise: 

e,{t + i) = {e,{t))r (2) 

Xiit + l)=Xi{t)+Viit + l) (3) 

where Vj(t + 1) is defined by ©, and {9i{t))r is the angle of the sum of the velocity vectors of 
neighbors of agent i: J2 Vj{t). In other words, {9i{t))r can be obtained by 

arctan( ^ sm{9j{t))/ ^ cos{9j{t))) (4) 

jeN,{t) j&N,{t) 

with some necessary regulations reflecting angles of [0,27r). 

Some recent results of Jadbabaie et al [6] have shown that under some connectivity conditions, 
the group will eventually move in a same direction. However, the model of [6] calculates {9i(t))j. 
by simply taking the average of angles of all neighbors. This brings convenience for mathematical 
analysis, but the system will sometimes exhibit counter-intuitive phenomena as they claimed in 
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their papeiQ. So this paper keeps the updating rule of the model proposed by Vicsek et al, even 
though the analysis result about synchronization of the model in [6] cannot be utilizeqj- 

One can think of many related questions: what is the way to help the group to synchronize 
if the group will not synchronize by self-organization? What if the group self-organize to a 
synchronized direction a which is not what we want? What if we want the group to fly to a 
desired destination? And so on. 

In this paper, we consider the case of adding one skill to control the collective behavior with 
the skill denoted as (jcq, ^o)- The ordinary agent (i = 1, . . . ,n) still keep the local rule as formula 
©-([I]) to update its heading and position. The only difference is that the neighborhood Ni(t) 
for agent i at time t will consider the skill: Ni(t) = {j\\\xj{t) — Xi{t)\\ < r,j = 0,1,2, ... ,n}. 
And the skill does not exactly obey the local rules as the ordinary agents do, its position and 
heading is decided by the control law u: 

(Xoit), Ooit)) = u{Xi{t), . . . ,Xn{t), ei{t), ..., 9n{t),t) 

Note that (jco,^o) may be subject to some constraints. For example, the constraint of xo(t+l) = 
xo(t) + vo(t + 1) will make the skill behave like an ordinary agent except the way it decides its 
own heading. Without any constraint, the skill can fly to anywhere with a 'deceptive' heading. 



u(t) 




Figure 1. The Soft control for the bii'd flocking model, by adding a skill. 
Two biids are liiilied if they are neighbors of each other at time t. 

The control law u will be different for different control purposes/tasks, such as to synchronize 
headings of the group (n agents), to keep the group connected or to dissolve a group, to turn 



^ This is because the zero angle is special than other angles in (0, 27r). The angles do not symmetrically behave. 
The result about synchronization for the model of [6] is not true for the model of Vicsek et al. Here is a counterexample: 
when V = 0, six agents are regularly put on the edge of a circle with radius of r, which form an ordinary hexagon. So each agent 
has three neighbors (because itself is also counted). The headings are 0, it/3, 27r/3, tt, 47r/3, Svr/S. This case will synchronize 
for the model in [6]. But it will not for the model we consider here, i.e. the Vicsek et al model without noise. 
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headings of the group (minimal circling), to lead the group to a destination (in a shortest time), 
to avoid hitting an object, etc. 

There are many problems to be considered in terms of control: under what conditions we can 
use the skill to control the state of the multi-agent system (the controllability problem); which 
kind of state information can be used for control (the observation problem), e.g., the skill can 
only see some nearest agents but not all other agents; how can one design the control law when 
the local rule and other information is not clear (the learning and adaptation problem), etc. We 
do not intend to answer all these questions in this paper. As a beginning, we will just show how 
it works for a specific case, which is to synchronize and turn the headings of a group. 

III. A Case Study 

Flocking of birds in airports is dangerous. How we drive them away? In this case, obviously 
we can not use centralized control, such as broadcasting orders to the birds to change to the 
desired flying direction. The current solution is to use cannon to shoot them away. But can we 
drive them away by soft controP. If the modified Bold model proposed by Vicsek et al works 
for real birds flocking, and if we can make a controllable powerful robot bird, can we use this 
robot bird as the skill and guide flight of the birds? In this section, we will study a simple but 
non-trivial related problem z . 

The, problem we consider here is: for a group of n agents with initial heading of 6*4(0) G [0, tt), 
1 < i <n, what is the control law for the skill, so that all agents will move to the direction of 
TT eventually? 

Suppose the local rule about the ordinary agents is known. Suppose also that the position 
xo{t) and heading 6*0 (t) of the skill can be controlled at any time step t. Suppose further that 
the state information (headings and positions) of all ordinary agents are observable at any time 
step. Now we propose an effective control law Uj3 which is defined as (see Fig. 2): 

'^In flocking behavior of real animals, more factors sucii as different sensory systems sfiould be considered. The Boid model 
is an abstract model which successfully simulates the flocking behavior. But no one has proved it to be true for real birds. So 
in this paper we just want to use a simple model to demonstrate how soft control works for self-organized multi-agent systems 
in general, but not to solve the specified airport birds problem. 
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TT 



(5) 
(6) 



where /3 G (0, tt) is a constant, and s{t) is defined as the 'worst' agent (or one of the 'worst' 
agents): s{t) = argmin{9i(t)}; 

l<i<n 

The intuition behinds m^ is to put the skill to the position of the 'worst' agent s(t) at each 
time step t and try to 'pull' it to the desired direction n. This greatly reduces the search space 
of positions and simplifies the control strategy. Note that we can not use 6o{t) = tt for all the 
time because: (A) in the case where ^s(t)(t) = and the all the neighboring ordinary agents of 
the skill have headings of zero degree, the skill with 9o{t) = tx can not change 9s{t)ii) according 
to the update rule (l2l); (B) to avoid the ill-case where both of the numerator and denominator 
in (lU are zer<£|. But in the final stages we do need ^o = tt to help the system to synchronize. 

In the sequel, we use A(t) = n — 9s{t)i't) to denote the distance between the angel of the 
'worst' agent and the objective angel n at time t. We now have the following main technical 
result of the paper: 



^In the ill-case, ^ sin{9j{t)) = implies 6j(t) = or tt for all j G Ni{t) because sina > for a G (o, tt). 
Consequently, by ^ cos{9j(t)) = we further conclude that half of the headings should be zero and the other half should 
be TT. So there must be at least one agent j 7^ such that 9j{t) — (since 6o{t) > for t > by the control law), which 
means 6s{t){i) = 0, and so we must have 60(f) — P £ (0, tt) if G Ni{t). However, this will contradict with the fact that 
ej{t) = or TT for all j G iV,(t). On the other hand, if ^ N.,{t), ^ sin(6lj(f)) = only holds when 6lj(f) = for all 

j6iV,(t) 

j G Ni(t) since obviously 6j{t) < n for all j — 1, . . . ,n and i > 0, but in this case we have ^ cos{6j{t)) > 0. So the 

jeiv.(t) 
ill-case will never happen in ^^. 
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Theorem 1. For any n > 2, r > 0, f > 0, and any initial headings and positions ftj(O) G [0, vr), 
Xi(0) G -R^, (1 < i < n), the update rule Q-© and the soft control law u/^ defined by dS])-® 
will lead to the asymptotic synchronization of the group, i.e., limi_^oo A(t) = 0. 

Proof. Because vr > ^o(^) > ^s(t)(^) > 0, and vr > 9i{t) > 9s{t)ii) for 1 < i < n, it is 
obvious by the update rule © that vr > 9i(t + I) > Os(t){'t) for 1 < i < n which implies 
that 9s(^t){t) < Os{t+i)(t + 1). Hence by definition of A(t), we have A(t) > A(t + 1) > 0, so 
A(t) will converge to some limit /i > 0. If /i > 0, then for any < £ < yU, we know that 
A(t) > 6 > holds for all large t. Consequently, by letting t — )• oo in the following Lemma 2, 
we have fi < ji — 5 which contradicts with 5 >0.So we can conclude /i = 0. This completes the 
proof of Theorem 1 . 

The main task now is to prove the following lemma. 

Lemma 2. If for some t > and e > 0, A(t) > £ > 0, then there exists a constant 5 E (0, e) 
which does not depend on t, such that A(t + n) < A(t) — 5. 

Proof At any time step t, the skill is affecting the agent s{t) with heading ^o(^)- We now 
proceed to analyze the lower bound to the angle change 9s{t)it + k) — 6s(t)i't) for I < k < n, 
which is denoted by r](k). For k = 1, the skill is affecting the agent s{t), so the worst case 
where r](l) has the smallest value is that the agent s(t) is surrounded by ra — 1 agents with 
heading ^s(i)(t) (see Fig.3-(a)). For k > 2, however, the skill may not affect the agent s(t), so 
the worst case where r](k) has the smallest value is that the agent s(t) is surrounded hy n — 1 
agents with heading 9s{t)it) again, but without the skill in its neighborhood (see Fig.3-(b)). Now 
by the definition of r]{k) we have 

Osit){t + k)>e,^t){t)+r]{k), (7) 

and a simple calculation will give 

min{arctan(sin/3/(n + cos /?)), arctan(sine/(n + cose))}, k = 1; 
V{k) = { (8) 

arctan(sin(77(fc — l))/(n — 1 + cos{r]{k — 1))), 2 < k < n; 

First, note that A(t) > e > and the function tjj{a) = arctan(sina/(m + cos a)), (m > 0) 
is monotonically increasing when a E [0, arccos(— 1/m)] and monotonically decreasing when 

a E [arccos(— 1/m), 7r]. 
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Second, to explain the definition of r]{l) in ([8]) we need to show that 

< r/(l) < arctan{sm{9o{t) - ^,(i)(t))/(n + cos(^o(t) - ^s(t)(t)))). (9) 

In fact, by the control law ^ and the expression of 77(1) in ([8]), the above inequality ^ is 
trivial in the case where 9s{t) < tt — /3. So we only need to consider the case where 9s(t) > tc — (3. 
In this case the control law is 9o{t) = n. We consider two subcases as follows: 

(i) 9Q{t) — 9s(t)it) > arccos(— ;i;). Since 9o{t)~9s{t)it) < vr— (vr — /3) = /3, by the monotonicity 
of the function V^(-), we know that the claim ^ is true. 

(ii) 9o{t) — 9s{t)it) < arccos(— ^). Since 9o{t) — 9^{t){t) = A{t) > e, by the monotonicity of 
the function ^(■) again, we know that the claim Q is also true. 

Hence, ^ holds in any case. 

Now, note that < ri{l) < 7r/2, and again by the monotonicity of the function ^(■), we have 
< riin) < ri{n -!)<...< r]{l) < n/2. 

Let us take 5 = ri{n) which is a finite positive value depending on n, /3 and e only. Obviously, 
we have 9s{t){t + n) - 9s{t){t) > S. 

Next, we consider the other agents: i = 1,2, ... ,n,i ^ s{t). 

Let A(A;) denote the set of agents whose angles are inside {9s(t){t),9s{t){t) + 5) at time step 
t + k, i.e., 

Aik) = {t\9sit)it) <9i{t + k) <9s^t)it) + 6, t = l,2, ...,n}. 

We now proceed to show that we must have A{n) = 0. 

First of all, as shown above, s(t) does not belong to A(A;) for I < k < n because 9s{t){t + k) > 
ds{t){t) + v{k) > 9s{t)it) + S. So we have |A(A;)| < n - 1 for 1 < A; < n. 

Therefore, at time step t + 1, there are at most n — 1 agents with angles which are less than 
9s{t){t) + 5. If A(l) 7^ 0, s(t+ 1) will be picked up from A(l). The skill will change the heading 
of agent s{t + 1). Because ^s(i+i)(t + 1) > ds(t)i't) and ^s(t)(^ + ^) > ds{t)i't) + ■r]{k) for any t, 
we have for k > 2, 

Os{t+i){t + k)> 9sit+i){t + 1) + v{k - 1) > ^.(t+i)(t + l) + S> 9,^t){t) + 5. 

So s(t + l) will not belong to A(A;) for any k G \2,n\, we have |A(A;)| <n — 2for:2<k<n. 
By repeating this argument, we get for 1 < d < k < n, 9s{t+d){t + k) > 6*3(4) (t) + ^ and 
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|A(/i;)| < n — d. So hy taking d = k = n, we have |A(r2)| = 0. This means that after at most n 
steps, A will be empty. 

Consequently, we know that 9i{t + n) > 9s{t){'t) + S for all i. Then we get 9s(^t+n){'t + n) > 
9s{t)i't) + 5 ^ A(t + n) < A(t) — 5. This complete the proof of Lemma 2. 

The snapshots of the relating computer simulation are showed in Figure 4. The video of demo 
and the simulation program can be downloaded from the project website [14]. 

Although our case study is a simple starting point, it shows that it is possible to change the 
collective behavior (headings) of a group by soft control. Based on this work, we are going to 
explore more complicated control problems, such as what if the energy of the skill is concerned 
(the energy of the shill is limited so that it can not 'jump' as in ([5])), what if the skill only can 
see locally, and what if the skill also needs to keep the group connected while turning the group, 
etc. 

To support this research, we have put a computer demo for soft control of the modified Bold 
model which is like a computer game on the project homepage [14]. After initially putting n 
agents on the two-dimensional space, the user can manually control the position and heading of 
the skill by keyboard and mouse. It is an open competition for the best control strategy. 




(a) t=0 



Cb)t=l 



(c) t=43 



(d) t=17S 



Figure 4. A sunuktiDrL for a group of n =50 and j^ = 7i/2. Tlie ordinaiy agents are represented by tlie 
small black dots with the hnie point to the direction they are moving. The ihill is the one with amow 
showing its heading direction and the circle showing the ciiment neighborhood radius of all agents. On the 
other hand, the circle of the shill indicates its efEectivie area. The system starts from a random initialization 
(a).AH>Oin(d). 



IV. Concluding Remarks 

The collective behavior of complex systems is the macroscopic feature that we concerned and 
studied in this paper. How we control or intervene in a multi-agent system without changing 
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the local rule of the existing agents is an important issue but has not been well recognized yet. 
The idea of soft control that we introduced and explored here appear to be an initial attempt 
to tackle this issue. We believe that our general soft control framework reflects a large class of 
control problems in complex systems. 

In this paper, we have restricted ourselves on the study of the modified Bold model which does 
catch certain key properties of many real world complex systems (see, e.g.,[3][4][6]). Without 
changing the local rule of the existing agents, we have added a skill in the distributed system 
and designed the control law for the skill carefully for the control purpose. Although the skill 
does not behave as the ordinary agents do, it is not recognized by the ordinary agents and is 
therefore still treated as an ordinary agent by them. This soft control idea works well as we have 
shown in the case study. We would like to emphasis that the idea of adding a skill is just one 
of the methods of soft control, and for different systems, they would be certainly different. 

There are many potential applications of soft control. Currently a project called Claytronics 
[11], a new form of programmable matter, is to study how to form interesting dynamic shapes 
and configurations by simple and local interactions of a billion micro-scale units. This research 
is in the category (II) of collective behavior. Using soft control to help to form shapes may 
release us from designing subtle local rules. Another possible case is to use soft control in the 
language diffusion model to guide evolution of language, and it might be able to save dying 
language [8]. A recent study in economics [9] shows that a small proportion of special agents 
who have preferences can dramatically alter the expected waiting time between norm transitions 
in a population of agents who repeatedly play the Nash demand game. If those special agents 
have feedbacks, how does the soft control help changing the transition of norms more efficiently 
? How we use soft control to avoid and handle panic of crowd [10] ... In conclusion, we need 
a theory of soft control to efficiently intervene in many complex systems. 
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